from sklearn_benchmarks.reporting.hp_match import HPMatchReporting

scikit-learn vs. ONNX Runtime

reporting = HPMatchReporting("onnx", config="config.yml")
reporting.run()

KNeighborsClassifier_brute_force

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=brute.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.307	0.070	0.0	0.018	-1	1	0.663	2.019	0.057	9.068	0.258	See	See
1	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.307	0.070	0.0	0.018	-1	1	0.663	0.023	0.002	811.551	84.234	See	See
2	KNeighborsClassifier_brute_force	predict	100000	1	100	0.304	0.010	0.0	0.304	-1	1	1.000	2.019	0.057	0.150	0.007	See	See
3	KNeighborsClassifier_brute_force	predict	100000	1	100	0.304	0.010	0.0	0.304	-1	1	1.000	0.023	0.002	13.467	1.470	See	See
4	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.303	0.010	0.0	0.018	-1	5	0.757	2.914	0.038	6.281	0.082	See	See
5	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.303	0.010	0.0	0.018	-1	5	0.757	0.027	0.003	688.017	71.212	See	See
6	KNeighborsClassifier_brute_force	predict	100000	1	100	0.322	0.006	0.0	0.322	-1	5	1.000	2.914	0.038	0.111	0.003	See	See
7	KNeighborsClassifier_brute_force	predict	100000	1	100	0.322	0.006	0.0	0.322	-1	5	1.000	0.027	0.003	12.116	1.274	See	See
8	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.946	0.097	0.0	0.019	1	100	0.882	2.085	0.007	9.088	0.054	See	See
9	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.946	0.097	0.0	0.019	1	100	0.882	0.020	0.000	927.888	22.862	See	See
10	KNeighborsClassifier_brute_force	predict	100000	1	100	0.311	0.010	0.0	0.311	1	100	1.000	2.085	0.007	0.149	0.005	See	See
11	KNeighborsClassifier_brute_force	predict	100000	1	100	0.311	0.010	0.0	0.311	1	100	1.000	0.020	0.000	15.217	0.595	See	See
12	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.422	0.065	0.0	0.018	-1	100	0.882	2.891	0.038	6.372	0.088	See	See
13	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.422	0.065	0.0	0.018	-1	100	0.882	0.023	0.002	802.013	75.012	See	See
14	KNeighborsClassifier_brute_force	predict	100000	1	100	0.308	0.004	0.0	0.308	-1	100	1.000	2.891	0.038	0.107	0.002	See	See
15	KNeighborsClassifier_brute_force	predict	100000	1	100	0.308	0.004	0.0	0.308	-1	100	1.000	0.023	0.002	13.425	1.269	See	See
16	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.276	0.030	0.0	0.018	1	5	0.757	2.053	0.005	8.904	0.026	See	See
17	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.276	0.030	0.0	0.018	1	5	0.757	0.019	0.000	939.501	7.647	See	See
18	KNeighborsClassifier_brute_force	predict	100000	1	100	0.308	0.007	0.0	0.308	1	5	1.000	2.053	0.005	0.150	0.003	See	See
19	KNeighborsClassifier_brute_force	predict	100000	1	100	0.308	0.007	0.0	0.308	1	5	1.000	0.019	0.000	15.833	0.371	See	See
20	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.268	0.160	0.0	0.018	1	1	0.663	1.185	0.009	15.421	0.182	See	See
21	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.268	0.160	0.0	0.018	1	1	0.663	0.020	0.002	935.084	114.903	See	See
22	KNeighborsClassifier_brute_force	predict	100000	1	100	0.308	0.005	0.0	0.308	1	1	1.000	1.185	0.009	0.260	0.004	See	See
23	KNeighborsClassifier_brute_force	predict	100000	1	100	0.308	0.005	0.0	0.308	1	1	1.000	0.020	0.002	15.756	1.946	See	See
24	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.702	0.015	0.0	0.004	-1	1	0.896	1.859	0.026	1.992	0.030	See	See
25	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.702	0.015	0.0	0.004	-1	1	0.896	0.008	0.003	453.774	178.141	See	See
26	KNeighborsClassifier_brute_force	predict	100000	1	2	0.240	0.005	0.0	0.240	-1	1	1.000	1.859	0.026	0.129	0.003	See	See
27	KNeighborsClassifier_brute_force	predict	100000	1	2	0.240	0.005	0.0	0.240	-1	1	1.000	0.008	0.003	29.417	11.562	See	See
28	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.737	0.014	0.0	0.004	-1	5	0.922	2.747	0.035	1.360	0.018	See	See
29	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.737	0.014	0.0	0.004	-1	5	0.922	0.009	0.002	436.032	109.162	See	See
30	KNeighborsClassifier_brute_force	predict	100000	1	2	0.241	0.004	0.0	0.241	-1	5	1.000	2.747	0.035	0.088	0.002	See	See
31	KNeighborsClassifier_brute_force	predict	100000	1	2	0.241	0.004	0.0	0.241	-1	5	1.000	0.009	0.002	28.101	7.054	See	See
32	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.775	0.010	0.0	0.004	1	100	0.929	2.005	0.048	1.882	0.045	See	See
33	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.775	0.010	0.0	0.004	1	100	0.929	0.003	0.000	1288.932	102.465	See	See
34	KNeighborsClassifier_brute_force	predict	100000	1	2	0.240	0.004	0.0	0.240	1	100	1.000	2.005	0.048	0.120	0.003	See	See
35	KNeighborsClassifier_brute_force	predict	100000	1	2	0.240	0.004	0.0	0.240	1	100	1.000	0.003	0.000	81.912	6.646	See	See
36	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.774	0.007	0.0	0.004	-1	100	0.929	2.792	0.044	1.351	0.021	See	See
37	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.774	0.007	0.0	0.004	-1	100	0.929	0.009	0.002	442.251	89.967	See	See
38	KNeighborsClassifier_brute_force	predict	100000	1	2	0.240	0.004	0.0	0.240	-1	100	1.000	2.792	0.044	0.086	0.002	See	See
39	KNeighborsClassifier_brute_force	predict	100000	1	2	0.240	0.004	0.0	0.240	-1	100	1.000	0.009	0.002	28.097	5.736	See	See
40	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.720	0.013	0.0	0.004	1	5	0.922	1.953	0.007	1.905	0.009	See	See
41	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.720	0.013	0.0	0.004	1	5	0.922	0.003	0.000	1281.258	97.265	See	See
42	KNeighborsClassifier_brute_force	predict	100000	1	2	0.240	0.005	0.0	0.240	1	5	1.000	1.953	0.007	0.123	0.003	See	See
43	KNeighborsClassifier_brute_force	predict	100000	1	2	0.240	0.005	0.0	0.240	1	5	1.000	0.003	0.000	82.777	6.493	See	See
44	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.763	0.013	0.0	0.004	1	1	0.896	1.074	0.003	3.503	0.017	See	See
45	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.763	0.013	0.0	0.004	1	1	0.896	0.002	0.000	2052.573	276.224	See	See
46	KNeighborsClassifier_brute_force	predict	100000	1	2	0.240	0.005	0.0	0.240	1	1	1.000	1.074	0.003	0.223	0.005	See	See
47	KNeighborsClassifier_brute_force	predict	100000	1	2	0.240	0.005	0.0	0.240	1	1	1.000	0.002	0.000	130.878	17.802	See	See

KNeighborsClassifier_kd_tree

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=kd_tree.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	120.957	0.000	0.0	0.121	-1	1	0.929	0.498	0.007	242.756	3.525	See	See
1	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	120.957	0.000	0.0	0.121	-1	1	0.929	0.003	0.001	44881.197	9190.423	See	See
2	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.575	0.059	0.0	2.575	-1	1	1.000	0.498	0.007	5.169	0.140	See	See
3	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.575	0.059	0.0	2.575	-1	1	1.000	0.003	0.001	955.579	196.885	See	See
4	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	119.319	0.000	0.0	0.119	-1	5	0.946	0.952	0.019	125.286	2.533	See	See
5	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	119.319	0.000	0.0	0.119	-1	5	0.946	0.004	0.001	33208.122	8184.547	See	See
6	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.606	0.110	0.0	2.606	-1	5	1.000	0.952	0.019	2.737	0.128	See	See
7	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.606	0.110	0.0	2.606	-1	5	1.000	0.004	0.001	725.417	181.372	See	See
8	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	118.638	0.000	0.0	0.119	1	100	0.951	5.743	0.091	20.658	0.329	See	See
9	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	118.638	0.000	0.0	0.119	1	100	0.951	0.003	0.001	34937.045	10109.491	See	See
10	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.584	0.045	0.0	2.584	1	100	1.000	5.743	0.091	0.450	0.011	See	See
11	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.584	0.045	0.0	2.584	1	100	1.000	0.003	0.001	760.832	220.557	See	See
12	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	129.118	0.000	0.0	0.129	-1	100	0.951	3.246	0.052	39.783	0.642	See	See
13	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	129.118	0.000	0.0	0.129	-1	100	0.951	0.006	0.001	23045.212	5135.412	See	See
14	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.590	0.065	0.0	2.590	-1	100	1.000	3.246	0.052	0.798	0.024	See	See
15	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.590	0.065	0.0	2.590	-1	100	1.000	0.006	0.001	462.238	103.658	See	See
16	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	127.413	0.000	0.0	0.127	1	5	0.946	1.726	0.025	73.802	1.081	See	See
17	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	127.413	0.000	0.0	0.127	1	5	0.946	0.002	0.001	78009.124	25546.517	See	See
18	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.628	0.098	0.0	2.628	1	5	1.000	1.726	0.025	1.522	0.061	See	See
19	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.628	0.098	0.0	2.628	1	5	1.000	0.002	0.001	1608.828	530.278	See	See
20	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	131.532	0.000	0.0	0.132	1	1	0.929	0.896	0.006	146.810	0.957	See	See
21	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	131.532	0.000	0.0	0.132	1	1	0.929	0.001	0.000	123527.635	44007.325	See	See
22	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.614	0.103	0.0	2.614	1	1	1.000	0.896	0.006	2.918	0.117	See	See
23	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.614	0.103	0.0	2.614	1	1	1.000	0.001	0.000	2454.924	879.945	See	See
24	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.040	0.002	0.0	0.000	-1	1	0.891	0.021	0.001	1.895	0.112	See	See
25	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.040	0.002	0.0	0.000	-1	1	0.891	0.002	0.000	16.073	3.255	See	See
26	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	1	1.000	0.021	0.001	0.245	0.012	See	See
27	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	1	1.000	0.002	0.000	2.082	0.416	See	See
28	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.040	0.000	0.0	0.000	-1	5	0.911	0.022	0.000	1.767	0.032	See	See
29	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.040	0.000	0.0	0.000	-1	5	0.911	0.002	0.000	17.600	2.586	See	See
30	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	5	1.000	0.022	0.000	0.230	0.008	See	See
31	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	5	1.000	0.002	0.000	2.296	0.345	See	See
32	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.060	0.000	0.0	0.000	1	100	0.894	0.033	0.000	1.841	0.021	See	See
33	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.060	0.000	0.0	0.000	1	100	0.894	0.001	0.000	98.080	28.648	See	See
34	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	100	1.000	0.033	0.000	0.158	0.006	See	See
35	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	100	1.000	0.001	0.000	8.401	2.470	See	See
36	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.061	0.000	0.0	0.000	-1	100	0.894	0.036	0.003	1.710	0.124	See	See
37	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.061	0.000	0.0	0.000	-1	100	0.894	0.002	0.000	29.122	5.314	See	See
38	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	100	1.000	0.036	0.003	0.146	0.011	See	See
39	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	100	1.000	0.002	0.000	2.480	0.457	See	See
40	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.040	0.000	0.0	0.000	1	5	0.911	0.021	0.003	1.923	0.260	See	See
41	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.040	0.000	0.0	0.000	1	5	0.911	0.001	0.000	65.843	19.284	See	See
42	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	5	1.000	0.021	0.003	0.251	0.035	See	See
43	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	5	1.000	0.001	0.000	8.591	2.529	See	See
44	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.039	0.000	0.0	0.000	1	1	0.891	0.019	0.000	2.128	0.026	See	See
45	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.039	0.000	0.0	0.000	1	1	0.891	0.001	0.000	65.391	19.378	See	See
46	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	1	1.000	0.019	0.000	0.281	0.010	See	See
47	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	1	1.000	0.001	0.000	8.650	2.580	See	See

HistGradientBoostingClassifier_best

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: learning_rate=0.01, n_iter_no_change=10.0, max_leaf_nodes=100.0, max_bins=255.0, min_samples_leaf=100.0, max_iter=300.0.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	n_iter	iteration_throughput	latency	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	HistGradientBoostingClassifier_best	predict	100000	1000	100	0.469	0.014	300	0.002	0.000	0.824	0.107	0.001	4.373	0.133	See	See
1	HistGradientBoostingClassifier_best	predict	100000	1000	100	0.469	0.014	300	0.002	0.000	0.824	0.016	0.001	29.144	1.510	See	See
2	HistGradientBoostingClassifier_best	predict	100000	1	100	0.388	0.010	300	0.000	0.388	1.000	0.107	0.001	3.619	0.094	See	See
3	HistGradientBoostingClassifier_best	predict	100000	1	100	0.388	0.010	300	0.000	0.388	1.000	0.016	0.001	24.120	1.192	See	See